Skip to content

ENH: add bx view static dashboard and pure-Python views layer - #131

Open
rok wants to merge 7 commits into
benchx-dev:mainfrom
rok:views
Open

rok wants to merge 7 commits into
benchx-dev:mainfrom
rok:views

Conversation

@rok

@rok rok commented Nov 20, 2026

Copy link
Copy Markdown
Member

Moves the static benchmark dashboard and derived-table pipeline from benchx-arrow into benchx as bx view (benchx.views), so users can generate deployable static sites or view local benchmark results in a browser directly from benchx.

Note: Built on top of #128 (trim-dependencies) to preserve optional PyArrow support.

Summary of changes

  • Pure-Python views package (src/benchx/views/):
    • Replaces chdb with pure Python stdlib for deriving flat tables (commits, points, changes, latest, movers) and static JSON view payloads (landing, series, sparks, history/<suite>, commits/<sha>).
    • PyArrow is imported lazily only when reading Parquet inputs (--store, --runs) or writing the optional Parquet marts under data/.
    • Supports three input modes:
      • Local Parquet store (default: store.parquet / $BENCHX_HOME, or --store PATH), collapsing multi-round attempts per commit, distinguishing dirty working trees (<sha>-dirty-<tree>), and enriching commit metadata via git cat-file --batch (--repo).
      • JSON result files (--results PATH...) via MemoryStore with zero PyArrow dependency.
      • CI runs/<sha>/<node>/{run.parquet, results.parquet} layout (--runs DIR), plus write_run() / run_schema() for CI publishers.
    • Bundles the generalized static HTML/CSS/JS dashboard under src/benchx/views/static/ (keeps CDN uPlot with graceful offline table fallback; vendoring uPlot tracked in Vendor uPlot assets in wheel for offline bx view #129).
  • CLI (bx view):
    • bx view (no --out): builds into a temporary directory and serves over local HTTP until Ctrl-C.
    • bx view --out DIR [--serve] [--open]: builds a deployable static site with cache-busted asset URLs (?v=).
  • Tests & docs:
    • Added tests/test_views.py (15 tests covering marts arithmetic, view payloads, Arrow-free JSON --results, local store round collapsing, --runs with logs and failed builds, and HTTP serving).
    • Updated tests/test_minimal_deps.py to verify import benchx.views does not import pyarrow.
    • Documented bx view in docs/benchx.md.

Verification

  • pytest tests: 844 passed, 6 skipped.
  • Verified parity of bx view --runs against benchx-arrow's existing chDB-built site/data/ across manifest.json, landing.json, series.json, sparks.json, history/*.json, and commits/*.json.
  • Verified node --check on src/benchx/views/static/app.js and confirmed uv build --wheel includes benchx/views/static/* and benchx/tests/test_views.py.

rok added 2 commits November 20, 2026 14:28
Move pyarrow>=19 into the optional benchx[parquet] extra, replace jsonschema[format-nongpl] with rfc3339-validator and rfc3986-validator, remove jsonpointer, and decouple JSON-only execution and comparison paths from Parquet storage.

Closes benchx-dev#116
- Catch FormatCheckerError in CLI main() and return EXIT_REFUSED (with test in test_refusals_not_crashes.py).
- Add --no-ingest to bx compare measure mode using MemoryStore (works for revisions, environments, and variants without pyarrow), validate flags before opening the store, and point missing-pyarrow hints to --no-ingest.
- Remove duplicated pyarrow ModuleNotFoundError wrapper in store.py (delegates to parquet.py).
- Replace module-level pyarrow skips with lazy Store imports and per-test requires_pyarrow marks across test files (+194 more tests run in the minimal no-pyarrow job).
- Narrow the minimal CI import check to pyarrow/jsonpointer and verify benchx's default vs parquet Requires-Dist directly on the built wheel.

@rok rok left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another 🤖 review.

Comment thread src/benchx/views/marts.py Resolved
Comment thread tests/test_views.py Resolved
Comment thread src/benchx/views/inputs.py Resolved
Comment thread src/benchx/views/inputs.py Resolved
Comment thread src/benchx/views/site.py Resolved
Comment thread src/benchx/cli.py Resolved
Comment thread src/benchx/views/marts.py Resolved
Comment thread src/benchx/views/marts.py Resolved
if isinstance(suites, list) and suites:
first = suites[0]
if isinstance(first, dict) and first.get("suite"):
return str(first["suite"]).rsplit("/", 1)[-1]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suite basename collision for nested Python suites: stripping everything before the last / matches Google Benchmark binary paths (release/arrow-bit-util-benchmark), but two nested Python suites sharing a filename (e.g. benchmarks/a/bench.py and benchmarks/b/bench.py) merge into one "bench.py" suite. Worth deciding whether to keep relative paths for non-binary suites (or strip only a leading build-dir prefix) in a follow-up.

"""
grouped: dict[tuple, list[dict]] = defaultdict(list)
for pt in points:
key = (pt["sha"], pt["node"], pt["suite"], pt["case"], pt["params"], pt["quantity"])

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-environment stores collide on the same commit: dashboard series are keyed by (suite, case, params, quantity, node) rather than series_fingerprint. If a store holds bx compare --profile environments --label env=a,b (or two different subject.configuration / protocol variants) on the same commit and host, _collapse_duplicate_points will pool both sides into one point. Worth noting in docs/benchx.md that bx view currently assumes per-commit revision history, or incorporating labels/configuration into the key in a follow-up.

ingest_problems = sum(r["ingest_problems"] for r in runs)
raw_points: list[dict] = []
for path in sorted(runs_path.glob("*/*/results.parquet")):
raw_points.extend(_read_parquet_points(path, distinguish_dirty=False))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scaling note for multi-year runs/ trees: read_runs_dir loads every */*/results.parquet into Python dicts on each build. Fine for local stores and months of CI runs, but once a project accumulates hundreds of thousands of rows we should benchmark build time and consider incremental/monthly rollups.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant